An adjunct test for discourse processing in MUC-4
نویسنده
چکیده
The motivation for this adjunct test came from an exploratory s tudy done by Beth Sundheim during MUC-3. This s tudy showed a degradation in correctness of message processing as the information distribution in the message became more complex, that is, as slot fills were drawn from larger portions of the message and required more discourse processing to extract the information and reassemble it correctly in the required template(s). The s tudy also suggested that systems did worse on messages requiring multiple templates than on single-template messages. These observations led us define the MUC-4 adjunct test to examine two hypotheses related to discourse complexity and expected system performance:
منابع مشابه
GE adjunct test report: object-oriented design and scoring for MUC-4
This paper reports on the results of the adjunct test performed by GE for the MUC-4 evaluation of text processing systems . In this test, we evaluated the effect of an object-oriented templat e design and associated matching conditions on the scores . The results indicate that the curren t MUC.{ "flat" templade design with cross-references closely approximates a true object-oriente d design . H...
متن کاملThe PRC PAKTUS System: MUC-4 test results and analysis
In commenting on PRC's performance in MUC-3, we reported [1] that the component of ou r system that could most improve performance was discourse analysis . The MUC-4 exercise has strongly confirmed that view. We added a new discourse module to PAKTUS, made very fe w changes to the other system components, and the result was significantly improved performance . This paper discusses our test resu...
متن کاملMessage Understanding Conference (MUC) Tests of Discourse Processing
Performance evaluations of NLP systems have been designed and conducted that require systems to extract certain prespecified information about events and entities. A single text may describe multiple events and entities, and the evaluation task requires the system to resolve references to produce the expected output.We describe an early attempt to use the results from an information extraction ...
متن کاملPRC Inc.: description of the Paktus system used for MUC-4
BACKGROUND The PRC Adaptive Knowledge-based Text Understanding System (PAKTUS) has been unde r development as an Independent Research and Development project at PRC since 1984 . It includes a core English lexicon and grammar, a concept network, processes for applying these to lexical, syntactic, semantic, and discourse analysis, and tools that support the adaptation of the generi c core to new ...
متن کاملLanguage Systems, Inc.: description of the DBG system as used for MUC-4
LSI's Data Base Generation (DBG) system is a syntax-driven natural language processing system that integrate s syntax and semantics to analyze message text. The goal of the DBG system is to perform full-scale lexical, syntactic, semantic, and discourse analyses of message text and produce a system-internal knowledge representatio n of the text that can serve as input to a downstream system or e...
متن کامل